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Controlled Multivariate Evaluation of Open and Traditional 
Education at the Junior High School Level: 
The First-Year Report 

Alan F, Sewell and Allan W. Dornseif 

Although undoubtedly some enterprising scholar could trace the ancestry of 
open education to the Greek akademeia (nevertheless, open education seems to be 
more socratic than platonic), the more remote of open education's contemporary 
ancestors was the educational philosophy of progress i vi sm of some four or five 
decades ago. The immediate parents of the open education movement (as it has 
come to be described), however, were the philosophies and experiences of the 
Informal classrooms of England, which emphasized the priority of student choice 
in the selection of learning activities and the student's conjoint role as 
learner- teacher. 

While in theory open education suggests a movement towa rd Certain educa- 
tional objectives, in practice it has been most distinctive as a movement away 
from traditional educational methods. Growing criticisms of and disenchantment 
with much of traditional education, both in England and in the United States, 
has led to a rather wide range of educational innovations, the communal ity of 
which is their departure from traditional methods. 

Awareness of the English experiments grew slowly during the early years of 
the last decade, but by the middle years of that decade, a considerable interest 
had developed in their possible applications to American education, Hawkins 
(1965, 1967) t for example, attempted to identify significant elements of open 
education in order to provide a theoretical base for an American version, but 
his primary focus was science education. The momentum of such Interest was 
substantially increased by the official sanction given the open education move- 
ment in England with the publication of the Plowden Report (1966) and by publN 
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cation in the United States of a series of germinal articles and a book by 
Featherstone (1968, 1971a, 1971b, 1971c), In these publications Featherstone 
not only explored the English principles and practices In detail, but he also 
provided a number of thoughtful suggestions bearing on the potential imple- 
mentation in American education. 

Still further impetus was provided by Silberman's Crisis in the Classroom 
(1970), which recommended elements of open education a promising alternatives 
to traditional education In the United States, Similarly influential in 
establishing philosophical and conceptual bases for American models were The 
ESS Reader (1970) and Open Education: The Informal Classroom (Rathbone, 1971). 

Possibly the earliest widespread application of concepts derived from the 
English informal classroom experiences took the form of the "free" schools 
which seemed to bloom everywhere in the United States toward the end of the 
last decade. These schools were and are characterized by a general 
la?sse2"fai re approach to planning, instead emphasizing freedom, spontaneity, 
and Individual choice. Many pointedly identified themselves as "alternative" 
schools, objecting to both the systemat i zat ion and the methodologies of tra- 
dftfona! education. In effect these schools deny the possibility of innovation 
and improvement within existing educational systems- 

Perhaps in response to such challenges, the past few years have seen an 
enormous development of conceptualization and implementation of various forms 
of open education in the United States. Most of this development has occurred 
within the educational "establishment" and appears generally to be a compro- 
mise between the extremes of informality advocated by "free" schools proponents 
and the fairly rigid structure of traditional education. Typifying this com- 
promise Is Howard's (I968) version of open education, the "Developmental Class* 
room," which holds that a free or open approach to education still requires a 
comforting Sense of structure. 

Official sanction in the United States was gained when late in the last de- 
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cade North Dakota adopted open education as a systematfc approach and, moreover, 
reorganized teacher training to ensureperpetuat r on of the adopted model. Simi- 
lar acceptance has been granted or is being considered in several other States, 
including New Jersey, New York, and Vermont. Massachusetts' Educational De- 
velopment Center has helped to organize a large number of open classrooms through- 
out the United States, following the basic pattern of the English informal 
schools. The Center also has developed materials and a useful bibliography 
(Barth S Rathbone, 1971) to aid in the planning and implementation of open edu- 
cate on programs. Educational Testing Service has also pub If shed (1970) a use- 
ful guide to esteb 1 i shment and evaluation of open education programs. 

How Open I s Open? 

Because open schools and open classrooms, as distinct from "free" schools, 
have typically been introduced within existing educational systems in the 
United States, compromises, as noted earlier, have been effected between the 
Concepts and methods of "pure" open education and the concepts and methods of 
traditional education. As a result there now exist hundreds of open classrooms 
and open schools which vary enormously. The open education movement has clearly 
not progressed sufficiently to have achieved any degree of standardization; 
indeed. It well may be that such standardization would prove to be the nemesis 
of open education. Nevertheless, until some consensus on openness is attaifir^jd, 
evaluation of the fundamental worth of this approach must necessarily proceed 
piecemeal. 

In recognition of this difficulty a number of attempts have been made to 
develop objective criteria of open education. Two of these attefiipts are par- 
ticularly noteworthy. Walberg and Thomas (1970 studied the writings of lead- 
tng British and American theorists of open education and, through content analy- 
sis, extracted eight basic or characteristic "themes." Subsequently they pro- 
vided (1972) an instrument Intended to objectify the degree of openness in an 




educational environment. These characteristics or "themes" are: (1) provision- 
ing for learning; (2) humaneness, respect, openness, warmth; (3) dfagnosis of 
learning events; {k) instruction, guidance, extension of learning; (5) evalua- 
tion of diagnostic information; (6) seeking opportunities for professional 
growth; (7) self-perception of the teacher; (8) assumptions about children and 
the learning process. Significantly, 25 of the 50 items of the questionnaire- 
observation schedule concern the "provisioning" theme, suggesting a preponderant 
emphasis upon variables of the physical environment. 

Traub et aK (1973) also used the theoretical literature of open education 
In an effort to identify critical characteristics. A 29- i tem rating scale, 
known as Dimensions of Schooling (DISC), was developed for use by teachers in re* 
porting aspects of school or classroom life. These items were chosen to reflect 
ten characteristics of open education. A subsequent factor analysis of 25 of 
the 29 Items produced six clusters of items: (l) individualization of instruc- 
tion, (2) student independence, (3) environmental flexibility, (k) nongradedness , 
(5) flexibility of student evaluation, (6) flexibility of curricular materials. 
The emphasis In the definition of open education implied by the DISC appears to 
be upon individualization, a characteristic which is, again by Implication, 
somewhat less directly important in the Wa 1 berg-Thomas definition. That is, 
scores on these two instruments might well be poorly correlated, suggesting 
that rather different criteria are involved; such a cross-validation study is 
now being conducted by the present authors. 

Clearly some obscurity of distinction between openness as an environmental 
characteristic and openness as a program characteristic exists, justifying the 
classification by Traub et a 1 . (1973) of "open space" schools and "open pro- 
gram" schools. The primary characteristic of the former is the architectural 
plan of the learning space and the kinds of learning and teaching behaviors 
necessitated and facilitated by absences of walls and fixed equipment. The pri- 
mary characteristic of the latter appears to be some degree of Individualization 




of Instruction, 

Somewhat related to this characteristic of open program school s is the In- 
dividual 1y Guided Education/Multi-Unit^ School (IGE/MUS) approach develbped in 
Wisconsin. More an attempt to increase individualization and team teaching, this 
approach nonetheless contains elements of a philosophy of open education: em-- 
phasls upon spatial arrangements, deployment of teachers, and se 1 f- gu i dance by ' 
the student* The degree of systemat i zat i on implicit in this approach reflects, 
however, the same kind of compromise noted earlier as generally applied In 
American models of open education. Also related in some degree to the philo- 
sophy of open education is the non-graded school; as noted previously, non- 
gradedness is one of the distinctive characteristics of open education in the 
view of Traub et al. (1973). 

Currently there is no single "American model" of open education. Instead 
there are many modeis^ each differing demonstrably from the others. They share 
in attempting to provide alternatives to traditional education, and very likely 
ttey const i tute simultaneous evolutionary experiments in educational reform. But 
the evolutionary struggle between traditional and Innovative education, and the 
struggle between competitive varieties of educational innovation will be settled 
less. by biological than by social processes of natural selection, and in parti- 
cular by the acceptances and preferences of the education-consuming public who 
support educational institutions and systems. Hopefully evaluative research 
m]\ enable that public to make informed choices. 

Some Relevant Research 

Because of the relative recency of open education In America and because of 
the plethora of somewhat dissimilar models, evaluation of open education is still 
fairly primitive. Indeed, an appropriate evaluation model has not yet been de- 
veloped* Most studies which purport to evaluate open education are conceptually 
or methodologically Imperfect, most commonly because of one or mere of the fol- 



lowing limitations: (1) lack of control groups pursuing traditional programs; 
(2) non«»random assignment of students to open groups and control groups; (3) 
use of groups at different schools, with a consequent confounding of socioeco- 
nomic variables; [h) insufficient definition of **openness'* as an independent 
variable; (5) insufficient rationale for selection of dependent variables; (6) 
use of Inappropriate analytical designs and techniques. Intrusion of such 
flaws Into the design, conduct, or analysis of results of these studies clearly 
prevents generalization of results and frequently justifies scepticism concern- 
ing results. 

At the present stage of evaluation strategies appT/cable to open education, 
it seems more appropriate to focus upon the nature of expected differences 
rather than upon the results of specific studies. Various studies have sought 
differences in academic achievement, in aspects of personality, in social be- 
havior,, and in self-concept. Burnham (1970, for example, compared Intelli- 
gence, ac^ ^ ygment . and behavioral differences of first-graders in open and tra^* 
di tional/^^&ol s. Sackett (1971) measured IQ., achievement, and self-concept of 
sixth-c^^fers In three different educational settings. Kil lough (1971) antici- 
pated fcteraction between sex and achievement as a function of open education. 
Warne r l|( 1971 ) compared op§n and traditional fourth grade classes in achievement, 
groupiira tJme, material usage, and classroom climate. Wi Ison et al . (1972) 
focused on measures of curiosity and productive thinking of students in open 
plan schools. Kennedy and Say (1971) compared second- through fifth-grade stu- 
dents of open and traditional schools in academic achievement. Carbonari (1970 
studied differences in personality factors as a function of length of experi- 
ence In an open school. The results of these studies are typically inconclusive 
and/or inconsistent with other studies. And because '^openness'' Is typically un- 
quantlfleuj no general izabi 1 ity can be derived from them. 

Hopefuf lyii with the development of instruments to measure ''openness** or to 
establish the degree of difference between open and traditional programs, a major 
O 
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limitation of such studies will be eliminated. One such study has already ap- 
peared and, within its own limitations, can well serve as a model for evaluative 
trends In open education. Having developed a means of quantifying openness, 
Traub et a 1 > (1973) proceeded to apply that means in an evaluation of student 
outcomes as a function of degree of openness, A number of Canadian elementary 
schools were classified according to architectural openness (''open architecture," 
**mixed architecture," or "closed architecture"), degree of program openness on 
the basis of DISC ratings, and character of student population (based on number 
of students employing English as a second language, hence, presumably repre- 
senting cultural homogeneity or heterogeneity). In terms of cognitive skills, 
no consistent pattern of advantages could be claimed for any given combination 
of architectural openness and program openness, at least for those schools in 
which the vast majority of students did not come from homes using English as a 
second language. However, in those schools in which more than 30% of the stu- 
dents were from homes using English as a second language (whom the authors 
seem to find similar to "so-called inner city kids"), a consistent pattern of 
cognitive superiority was found in traditional programs. Results of non-cogni- 
tive measures again showed no clearly consistent pattern, although the investi- 
gators report that many of the differences between groups are in the direc- 
tions which would be predicted by open education proponents, and that these dif- 
ferences are more clear-cut and more striking for older students (11-year-olds) 
than for younger students (8-year-o 1 ds) . 

Traub et aJ , acknowledge several of the deficiencies of their study, yet 
It quite likely represents the most careful and conscientious research to date 
in the area of open education. Of particular merit are the rather large number 
of schools involved, the classification of degrees and kinds of openness, and 
the number of variety of dependent variables considered. Nevertheless, the re- 
sults are more provocative than definitive, even within the limitations of the 
population studied, 
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Although the English experiences in open education and large-scale adoption 
of American variants seems to fmply an established superiority of open education 
over traditional education, the typically equivocal results of recent research 
recommend a more cautious approach. There appears to be no urgency to a belief 
that **openness" is necessarily desirable for all types of students, nor for al! 
ages of students, nor for all educational levels. It well may be, as results of 
the study by Traub et al . (1973) suggest, that for certain students traditionally 
structured education yields more satisfactory outcomes. With such a possibility 
in view, the present study was undertaken. 

Origin and Operation of the OSCAR Project 

This evaluation project was conducted at the 0. W. Huth Upper Grade Center, 
School District l62, Matteson, Illinois, When it became necessary to construct 
an addition to the existing Upper Grade Center building, a decision was reached 
to Implement an experimental open education program which would include seventh 
and eighth grade boys and girls in a single large classroom and which would be 
staffed by an interdisciplinary team of teachers. 

The District, located In south suburban Cook County, includes all or parts 
of five communities which encompass quite a wide socioeconomic range. Approxi- 
mately 3,^00 children attend the Distrrct^s eight schools. Of these, some 2,600 
children attend seven primary schools, and about 800 attend the District's 
single junior high school (the Upper Grade Center). Children attend local 
schools through sixth grade and then complete seventh and eighth grade at the 
Upper Grade Center, Hence the District's heterogeneity is fully expressed only 
at the junior high school level. 

The open classroom of the Upper Grade Center was planned and designed as 
part of a total remodeling and enlargement program, A space equivalent to that 
of five traditional classrooms and their adjacent hallway was allocated to the 
open classroom: a single, undivided area approximately 1^0 feet long and kO feet 



wide. The fortuitous acronym OSCAR was soon applied to the open classroom pro- 
ject, memorializing a former District superintendent and representing "Open 
Space for Conceptualizing Attitudes and Responsibilities." 

The OSCAR program, as developed, called for staffing by four teachers and 
two teacher-aides for a total of 1^0 seventh- and eighth-graders. One teacher 
was chosen from each of fogr major discipline areas: language arts, mathematics, 
science, and social studies. These teachers were chosen on the basis of ex- 
pressed Interest in and probable capability for the OSCAR program. During the 
summer preceding inauguration of the program, they attended a ten-day multi- 
talent development training program which emphasized the discovery and develop- 
ment of talents unique to each student. The OSCAR concept required these teachers 
to plan and work together in content areas and to develop a mul t id i sci p 1 i nary 
approach to subject matter. 

Initiation of the open classroom project was planned for the 1972-73 school 
year. In the Spring of 1972 two groups of children were randomly chosen: an 
OSCAR group and a Control group, with 1^0 chMdren in each group. Selection of 
the control group was based upon an intention to provide clear and continuing 
evaluation of the open classroom project. Half of each group was chosen from 
the Upper Grade Center's existing enrollment of then-seventh-graders, and half 
was chosen from the District's population of then-sixth-graders (who would at- 
tend the Upper Grade Center as seventh-graders during the 1972-73 school year). 
Each group included 35 seventh grade boys, 35 seventh grade girls, 35 eighth 
grade boys, and 35 eighth grade girls. The random assignment procedures were 
designed to ensure proportional representation of the entire District on the 
basis of each primary school's contribution to the junior high school population* 

OSCAR students were informed of the i r ass i gnment during Spring, 1972. Con- 
trol group students were never specifically informed of their function but were 
identified at times of assembly for testing as the principal's "Special Group'* 
in an attempt to control for Hawthorne effect. 

O 
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Early months of the OSCAR program were somewhat confused, primarily as a re- 
sult of construction delays and the newness of the program. The OSCAR teachers 
found it necessary to modify their plans with respect to multt-talent development 
and multldJ scipli nary planning. The school day was instead organized around the 
four major disciplines in the morning, with the amount of time devoted to each 
discipline determined cooperatively by the teachers. Projects and modified con- 
tract assignments were the basic activities of students in all subjects except 
mathematics, wherein an individualized skill development materials kit was used. 
During the afternoon, OSCAR students were scheduled into other school programs, 
such as physical education, home economics, industrial arts, music, and art, 
While most of the OSCAR group was out of the open classroom during this timCi 
about 25% of the students remained in the room to complete individual assign- 
ments» work in small study groups, or receive individual or small group tutor- 
ing. 

Control group students, meanwhile, pursued the ''t rad i t tonal '* departmental** 
i^ed curriculum and schedules of the Upper Grade Center, changing rooms and 
classmates at the end of each class period. These students were never together 
as a group except when assembled for testing purposes associated with this study. 
No attempt was made to encourage interdisciplinary teaching or the introduction 
of any unusual teaching techniques among non-OSCAR teachers* For purposes of 
this study, however, four non-OSCAR teachers, one in each of the four primary 
discipline areas, were selected as "control" teachers without their knowledge; 
this was done to permit some comparisons of teacher behaviors. 

While no attempt was made during the course of the study to evaluate the 
relative "openness'* of the OSCAR program — as, for example, by the DISC (Traub. 
et al . (1973) or by the Walberg and Thomas (1971) scale it is clear that the 
OSCAR program was designed to implement a fairly limited degree of "openness" as 
compared to the informality, freedom, and structure lessness of "free" schools. 
Nevertheless the freedom afforded the OSCAR students was substantially greater 
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than that accorded the control students. The OSCAR program was intended to incor- 
porate as many of the general characteristics of "open education" as seemed ap- 
propriate to the needs and abilities of students, teachers, and parents. 

This evaluation study was primarily focused upon student outcomes and was 
Intended to determine (1) whether the open and traditional programs produced over* 
all differences in these outcomes, and (2) whether student outcomes differed as 
a function of student characteristics, such that criteria for assignment of in- 
dividual students to a given program could be formulated. 

Method 

The basic design strategy of this evaluation project was to establish a re- 
search model which would permit continuing, definitive comparisons in many out- 
Come areas. The first-year plan, then, amounted to a "shotgun" approach to 
evaluation', hopefully to provide direction for subsequent, more specific evalua* 
tive efforts within an established model. 

A variety of measures, in the general areas of academic, personal, and so* 
ciaJ development, were used. The Stanford Achievement battery, long employed 
in. this District, was used to measure academic achievement in the areas of para- 
graph meaning, spelling, language, arithmetic computation, arithmetic concepts, 
arithmetic application, social studies, and science. Baseline data for the 
eighth-graders were available from the final testing of the previous academic 
year, but the seventh-graders had (as sixth-graders) completed a different form 
of the battery. The entire battery was administered twice during the evaluation 
year: at midpoint and at the end of the year. 

Various aspects of personal and social adjustment were measured by the Bell 
Adjustment Inventory. This instrument, although standardized for high school 
students, was chosen because of its reported reliability and its wide use among 
student populations. The Inventory was administered three times: near the be- 
ginning of the year, at midpoint, and near the end of the. year. Data from the 
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first administration, however, were discarded because (1) through administrative 
error a large number of the control subjects did not conplete the Inventory, and 
(2) a Subsequent study of the comprehens i bi 1 i ty of the items by the subjects led 
to the elimination of approximately 25 items which were not understood by the 
subjects. To prevent difficulties occasioned by the relatively advanced reading 
level of the Inventory, all items were tape-recorded for the second and third 
administrations; subjects were not given the Inventory booklets. 

Other Instruments were administered only once, near the end of the school 
year* These instruments were the California Psychological Inventory (Gough, 
1957t revised edition 1969), which provides data in 18 areas of persona 1 ity char- 
acteristics; the Piers-Harris Chi I dren ' s Se 1 f Concept Sea le, which provides a 
single measure of self-concept; the School Inventory (Bell, 1963), which measures 
attitude toward school; and a measure of ''locus of control'* (Rotter^ 1966) to 
provide data on the Internal-external orientation of the student. 

On three occasions during the year, the four OSCAR and four "control** 
teachers were observed by a trained observer who recorded behaviors according 
to Flanders' (1966) categorical analysis of classroom interactions. Finally, 
at the end of the year the four OSCAR teachers were asked to rate each OSCAR 
student on a five-point scale on each of four dimensions of performance: attitude, 
knowledge, skills, and sociability. The fol lowing behavioral definitions were 
provided to each teacher: 

Attitude: Student displays positive attitudes toward school, 
teachers, other school personnel, and other students. 
Knowledge: Student demonstrates mastery of academic content 
appropriate to his/her age, grade level, and apparent ability. 
Skill: Student demonstrates application of academic content 
within school and display? ability to apply academic content 
in non-academic settings. 

Sociability: Student- demonstrates respect for the rights and 
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feelings of others and demonstrates ability to work ef- 
fectively and cooperatively with others. 
Teachers were instructed to evaluate eacli student independently of other stu- 
dents and to use each rating dimension independently of the others. 

Lists of the control group students were provided to all teachers in the 
school's departmental program, and each teacher was asked to rate each student 
who had been in his classes during the year. This procedure was necessary for 
the control group since the four "control'* teachers had not instructed all the 
control group students during the year. 



Resul ts 

Two varieties of data analysis were conducted: absolute analyses , which fo- 
cussed Upon whether any absolute differences in outcome me^^sures could be demon- 
strated between the OSCAS and Control groups; and relative ar<alyses , the ob- 
ject of which was to detect differences in the nature of program outcomes. For 
the first of these varieties, the primary independent variable was the grouping 
variable (OSCAR vs. Control); independent variables of secondary interest were 
grade level (seventh vs. eighth), and sex (male vs. female). For the second 
variety of analysis, the single independent variable was program outcome as ei- 
ther ''success** or "failure." 

Since three major evaluative instruments employed in this study (Stanford 
Achievement battery, Bell Adjustment Inventory, California Psychological Inventory) 
consisted of numbers of intercorrelated subscales, data from these instruments 
were analyzed through multivariate analysis of variance techniques. Where over- 
all significance was demonstrated, post-hoc uni vari ate _Fs and step- down ^Fs were 
computed. Data from single-scale instruments wer^ analyzed through appropriate 

univariate analysis of variance models. In all Li s analyses were based upon 

i / 

raw scores rather than grade- level equivalents, /standardized scores, or the like. 
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Absol ute Analyses 

For purposes of economy, results will be summarized for each instrument. 
Unless otherwise noted, scales not mentioned did not significantly differentiate 
the two groups, 

Stanford Achievement Battery, The multivariate £ for Group was significant 
(F » 6.009, d^ = 8/201 , £< .001) . Two significant subscale differences were 
found by post-hoc analyses: Social Studies (F = 3»729, df = 1/208, £^<0l) and 
Science (f = I'f.S^^, df = 1/203, d<,001). In both cases the OSCAR group out- 



performed the control group. ThC multivariate _Fs for Grade Level and for Sex 
were also significant; as would be expected, eighth-graders outperformed sevenths 
graders, and females outperformed males in those areas in which differences oc- 
curred. 

The mult Ivariate _F for the Group x Grade Level Interaction was also signi- 
ficant (F = 2.713, df = 8/201, £^.0l). Post-hoc analyses showed that in Spell- 
ing and In Language, the seventh grade OSCAR students outperformed other sub- 
groups, with the eighth grade Control students second-best. In Arithmetic Com- 
putation and Arithmetic Concepts, the eighth grade Control students excelled, 
with the seventh grade OSCAR students second-best. No other Interactions were 
signif IcanV. 

Bell Adjustment Inventory. The multivariate _F for Group was significant 
(F = 3.Sk3, df = 6/206, £<.0l). Post-hoc analyses showed that significant dif- 
ferences occurred on the Subml ssi veness-Se 1 f-Assert i on scale (F = 6.^17. jdf = 
1/206, £<.01) and the Masculinity-Femininity scale (f = 11.073, df = 1/206, 
£<.001); In both cases higher scores were achieved by the Control group stu- 



The multivariate f for Grade was significant (F = 2.310, df ^ 6/206, £<.05). 
The only scale contributing significantly to this difference was the Submissive- 
ness-Self-Assertion scale (F = ^.515, df = 1/206, £^ .05)\ here the seventh- 
graders achieved higher scores than the eighth-graders. 
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As would be expected, the multivar icite F for Sex was also significant = 
39.7^1, df = 6/206, £<.0O1). Post-hoc analyses showed females to score higher 
fn Emotionality than males, and males to score higher than females in Masculinity- 
Femininity. None of the interactions was significant. 

California Psychological Inventory. Of the 18 scales of the Inventory, only 
one differentiated the two groups: Sense of Well-Being (univariate^ = 5»010, df^ 
= 1/165, £<.05). The Control mean for this scale was higher than the OSCAR 
mean. 

Piers-Harris Children's Self Concept Scale. This scale provides a single 
score; hence results were analyzed by ANOVA. The J[ for Group was the only sig- 
nificant finding (F = 7.215, df = 1/208, £^.01). The Control mean was higher 
than the OSCAR mean. 

The School Inventory. ThJs attitude inventory yields a single score; hence 
data were analyzed by ANOVA. None of the effects attained statistical signifi- 
cance. 

Locus of Control . Here, too, a single score is produced, and ANOVA was 
employed. Noine of the effects was significant. 

Relative Analyses 

The primary objective of the several relative analyses conducted was to per- 
mit differential characterization of the two educational programs and to identify 
student characteristics appropriate to each program. The paucity of significant 
findings through absolute analyses was» conceivably, due to the presence of stu- 
dents la each group who might have functioned significantly differently in the 
other group. Hence the focus of interest In the relative analyses was the stu- 
dent's relative performance within his group. 

Teachers' ratings of OSCAR and Control students were chosen as criteria for 
the relative analyses. In the case of the OSCAR teachers, each teacher's four* 
dimension rating of students was converted into a z distribution for that teacher, 
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and on the basis of this distribution a mean ^ rating across the four areas was 
obtained for each student. An overall mean ^ rating for each student was de- 
rived from the mean z rating provided by each teacher; this final rating, then, 
was the end-product of 16 separate ratings by the four OSCAR teachers, corrected 
by the ^ transformation for differences in rating styles. A similar procedure 
was followed for Control group students, except that in some cases fewer than 
four teachers rated a student and, in consequence, each such student was dropped 
from the analysis; and in some cases more than four teachers rated a student, 
but only the *^first" four ratings were used. 

On the basis of these mean z ratings, each group was then divided as near 
the median as possible; those above the median were classified as "successful*' 
students, while those below the median were classified as "unsuccessful" stu- 
dents. These admittedly arbitrary classifications permitted analyses of the 
"success- fal lure" characteristics of each group and, hence, each program. Only 
students who completed all measures employed in this study were included in 
these analyses; the totals thus available were insufficient for analysis by sex 
and grade level. Hence Group was the only independent variable employed In 
these analyses. 

In the first analysis "successful" students were compared to "unsuccessful" 
students, regardless of group. Subsequent analyses compared various subgroupings 
of "successful" and "unsuccessful" students. Each analysis was a one-way multi- 
variate analysis of variance, followed, where indicated, by univariate £ tests. 
The results of these analyses are shown In Table I. 



{nsert Table 1 about here 



Teacher-Related Measures 



The study provided two types of measures of teacher characteristics. Flan- 
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ders analyses of observations of teachers' behavior patterns in classroom inter- 
actions were conducted on three separate occasions. None of these analyses 
showed significant differences in patterns displayed by OSCAR teachers and by 
Control teachers. Review of the raw data showed that very substantial between- 
teachers variance occurred In both groups, and, with so small a sample size 
(four teachers In each group), very dramatic differences would have been required 
in order to attain statistical significance, 

A second approach to the analysis of teachers" behaviors and styles was 
suggested by the student ratings provided by each teacher. Because each of a 
number of teachers had rated a relatively few students In the Control group, com- 
parisons of rating characteristics were not possible. However, each of the four 
OSCAR teachers had rated the same 131 OSCAR students, and it was thereby possible 
to compare ratings* Intercorrelations of the teachers' mean z rating for each 
student are Shown in Table 2, 



Insert Table 2 about here 



Although grade level and sex were not included as independent variables in 
the relative analyses of program outcomes, it was possible to analyze the contri- 
butions of these factors to the OSCAR and Control teachers' ratings. Tests of 
differences In mean j ratings of seventh and eighth grade students in the •'suc- 
cess'* and "failure" subgroups were conducted. For the OSCAR "success" subgroup, 
the difference was significant (t « 7.670, df = 66, £^,001); seventh and eighth 
grade OSCAR "unsuccessful" students did not differ slignif icantly in mean ratings. 
The polnt-bl serial correlation coefficient between grade level and "success'* in 
the OSCAR group was .213 (£^«01)« Grade level was not found to be significantly 
related to mean z rating In ths Control group. 

The relation of sex to "success" and "failure" In each group was tested by 
a chUsquare test of Independence. Within the OSCAR group these two variables 

ERIC 



18. 

2 

were found to be significantly related ()[.. = 8,98, jdf 1 , £<'.01). The Goodman- 
Kruskal Index of predictive association (Hayes, 1963, pp. 608-609) was computed 
as ,183; that is, knowing the sex of the student reduces the probability of error 
in predicting success- fai lure by 18.3%. A point-biserial correlation coefficient 
between sex and mean z rating was also calculated for the OSCAR students; the 
obtained coefficient was .^07 (£^.001), showing good agreement with the Good- 
man-Kruskal index. The mean difference between male and female z ratings is 
not» however, statistically significant. The chi-square test between sex and 
success- fai lure within the Control group showed that in this group the two vari- 
ables are independent (X = 1.1^, df = I, £>,05). 

Discussion 

Educational innovation, like any other kind of innovatton, must justify it- 
self on either or both of two bases: (1) that it accomplishes traditional ob- 
jectives more efficiently; (2) that it accomplishes new (non-traditional) and de- 
sirable objectives. Hence the accomplishments of the OSCAR program must be 
evaluated finally in terms of those two bases. 

Tradt ti onal Object ives 

Mastery of academic content is probably the most "traditional" of educa- 
tional objectives* Laypersons and probably most professional educators would 
agree that the fundamental objective of education is to "learn the three R's." 
At the high school and junior high school levels the traditional emphasis upon 
academic achievement is demonstrated by departmental organization, with each con- 
tent area staffed by teachers possessing special subject-matter competences. It 
is quite iikely that any educational program which demonstrated decrements in 
academic achievement would rapidly be rejf'Cted by public and educators alike. 

Fortunately* the OSCAR program has demonstrated no such decrements; indeed, 
in two of four major subject-matter areas (social studies and science) the open 

O 
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education program appears to have demonstrated superior results. Performance of 
students In both groups exceeded national norms to some degree; mean grade level 
equivalents over the eight areas evaluated by the Stanford Achievement battery 
were as follows: seventh grade OSCAR 8.14, seventh grade Control 7.22, eighth 
grade OSCAR 8.49, eighth grade Control 8.S5. Hence the superior academic per- 
formances of the OSCAR students cannot be attributed to any obvious deficiencies 
In the traditional program. 

The Interactive effects of grade level and type of program upon academic 
achievement tempt a conclusion that open education programs may be better suited 
to younger students. Other findings of this study, however, reduce the urgency 
of such a conclusion. In any event, it can quite safely be concluded that the 
open education program of this school district is quite compatible with tradi- 
tional educational objectives and may, in fact, promote a more efficient accom- 
plishment of those objectives. 

Non-Traditional Object i ves 

In terms of traditional objectives, success in school can be measured as a 
direct function of academic achievement. But any program which proclaims Itself 
"alternative" or "innovative" must, by definition, imply a different kind of 
success. Although the philosophies and theories of traditional education gener- 
ally agree that the objectives of education should extend beyond the purely cog- 
nitive realm, the theoretical literature of open education much more strongly 
emphasizes non-cognl t I ve objectives. A strong implication exists that "success" 
in open education includes the student's performance and development In attl- 
tudlnal and social areas as well. Since these accomplishments are not necessarily 
attained at the expense of cognitive skills, open education Implies a wider, more 
encompassing definition of "success." 

To be sure, a definition of "success in school" Js not readily achieved. 
Prior to adoption of the teacher rating strategy employed in this study, the 
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four open classroom teachers and the four "control'* teachers were asked to pro- 
vide lists of behavioral objectives which governed their teaching. The lists 
provided by the two groups of teachers were essentially indistinguishable. While 
neither list was particularly noteworthy as an example of behavioral objectives, 
both Included items which clearly extended beyond the purely cognitive realm. 

The rationale of the teacher rating strategy held that "success" within a 
given program could, perhaps, best be defined by those student characteristics 
which significantly differentiated "successful" from "unsuccessful" students. 
That ts, the program's (or teacher's) objectives could be inferred from the dif- 
ferential characteristics of those students who did and who did not meet those 
objectives. 

The patterns of significant differences shown in Table 1 make ft clear that 
•'success'* Is not differently defined by open classroom teachers and traditional 
teachers: In both the major emphasis is upon academic achievement. The student 
who Is •'successful" In the traditional program is likely to be equally ''success* 
fgl'* in the OSCAR program. And there is no reason to believe that the student 
who Is •'unsuccessful" in the OSCAR program will attain "success" in the tradi- 
tional program. The emphasis upon academic achievement in both prpgrams is pro- 
bably also reflected in a common personality characteristic of the ''successf ul" 
student: responsibility. The lack of difference in student outcomes in the two 
programs supports a belief that the objectives of the two programs do not dif- 
fer; whether intentionally or not, the open classroom teachers have been pursuing 
conventional , traditional objectives. 



Program Homogeneity 

No undertaking, whatever its nature, can be expected to reach its objectives 
unless Its participants agree on those objectives. Hence it Is important to con- 
sider the degree to which agreement on program objectives has been achieved. 
Again the teachers* ratings of students are of use: agreement by the teachers as 
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to the relative "success*' of each student, reflected in the intercorrelations of 
ratings shown in Table 2, can serve as an indicator of inter-teacher agreement 
on program object ives . and, thus, of the homogeneity of the program. 

The intercorrelations of Table 2 provide a somewhat ambiguous picture. While 
most of the Intercorrelations are uncomfortably low, ratings by two of the four 
teachers are quite highly correlated. Several possible explanations are avail- 
able; (1) these correlations actually are conditioned by the similarities and 
dissimilarities of the subject-matters areas themselves; (2) the correlations re- 
flect fairly broad academic abilities of the students, in the sense that a stu- 
dent who does well in one area is 1 i ke ly to do we 1 1 in other areas; (3) the cor- 
relations do actually reflect teacher communal ities in objectives and methods. 
The sheer magnitude of the higher correlations of Table 2 tend to make the first 
two of these explanations untenable; the correlations substantially exceed the 
magnitude of correlations between Stanford subtests in the present study. Thus 
It appears that to a considerable degree at least two of the four teachers have 
been able to reach agreement on objectives, whether or not that agreement has 
ever been made explicit. 

Unfortunately It was not possible to calculate similar Intercorrelations 
for the control students* teachers, due to the relatively small number of stu- 
dents rated by each of the school's departmental teachers. Inspection of the 
ratings^ however, suggested wide variability and rather poor agreement In ratings. 
This lends some credence to a belief that inter-departmental programs generally 
display rather poor homogeneity. 

Special Problems of the Open Classroom 

One of the original purposes of this evaluation study was to derive dif- 
ferential profiles of "successful"* OSCAR students and ••successful"* traditional 
program students. Results of the study, however, demonstrate the futility of 
such an dttenpt at the present time: successful students in the two programs do 
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not differ because the programs do not, as yet, differ. Certain of the findings, 
however, paradoxically lend optimism to an expectation that special problems of 
the OSCAR program will require subsequent program differentiations and establish- 
ment of differential program objectives. 

As noted previously, ^'success** in both the open classroom and the tradition- 
al classroom depends primarily upon academic achievement. In the open classroom 
but not In the traditional classroom, "success'* is also positively correlated 
with sex (females having the advantage) and with grade level (seventh-graders 
having the advantage). "Successful" open classroom students also are character- 
ized by "communa I i ty," which is essentially defined as a sense of belonging to a 
group and pursuing group objectives; a relative absence of this characteristic 
Is found In "unsuccessful" open classroom students, but "communa 1 i ty" does not ap- 
pear to be related to success in the traditional program. The "successful" OSCAR 
student also differs from his counterpart in the traditional program in havfng 
a lesser "sense of well-being" and a significantly lower self-concept. And, 
finally, "successful" OSCAR students are more internally controlled than "un- 
successful" students in this group. 

Taken together, these findings present 3 fairly consistent portrait of the 
"successful" OSCAR student and, hence, of desirable student characteristics and 
behaviors within the open classroom. The student "most likely to succeed" in 
the OSCAR program's first year was a hard-working, self-disciplined, achieve- 
ment-oriented seventh grade girl; she cooperates with her teachers, yet has a 
sense of personal dissatisfaction. 

The data-gathering processes of this study were occasionally supplemented 
by Interviews with teachers and students of the OSCAR program. The results of 
these interviews tend to support the general validity of the preceding descrip- 
tion of the "successful'* student. Teachers' comments typically focussed upon 
problems of classroom management, discipline, order, and noise; individual. 
Students most easily recalled were usually "problem^* students, and the problems 
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were usually disciplinary in nature. Students, too, commented upon classroom 
nofse and an atmosphere of apparent confusion. Indeed, as the year-end neared, 
a number of seventh-graders petitioned the school's principal for transfer to 
the traditional program for the following year. While inevitably some of the 
reasons for these requests concerned feelings about individual teachers, a large 
number of the given reasons dealt with the student's difficulties in adjusting 
to the atmosphere of the open. classroom and its consequent demands upon self- 
discipllne* Many of the students remarked, for example, that they were unable 
to complete assignments unless, as more commonly occurs in the traditional pro- 
gram, an Immediate due-date is specified. Interviews also indicated that to 
some extent a morale problem has been generated within the OSCAR program through 
negative comments by non-OSCAR teachers; some of these teachers, apparently un- 
convinced of the program's worth and envious of the space and facilities allo- 
cated to it, had suggested that the "special" quality of the OSCAR program was 
that tt was remedial! 

Thus it appears that this open education program and its Installation may 
have presented challenges to the entire school population, students, teachers, 
and administrators: challenges which were unexpected and with which the school 
was unprepared to cope. While the space allocated to the OSCAR program, and the 
number of chi 1 dren assi gned to that space, approximates the school-wide average, 
the space itself demands new and different student and teacher behaviors. Noise, 
and a degree of apparent confusion are inevitable consequences of housing so 
large a group of students In so large a space, and the teacher's probable first 
concern Is with attaining order and discipline. This concern seems clearly re- 
flected in the personal characteristics of the "successful" OSCAR student. It 
is to be expected that as their experience accumulates the OSCAR teachers will 
develop coping strategies which will enable them to focus more effectively upon 
their students' non-cognitive development; that is, ''success" will be redefined 
to expand concepts of responsibility and sociability and to de-emphasize concepts 
O 
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of order and discipline. The first year of this program has clearly demonstrated 
that academic standards can be maintained. With the teachers' increasing confi- 
dence and Competence, the potential contributions of open education to the stu- 
dent's personal and social development can more meaningfully be assessed. 

The findings of this study suggest some general observations pertinent not 
only to this particular open education installation but probably equally relevant 
to other present and contemplated programs: 

1. Clearly a critical requirement of an open education program is more 
effective preparation of the teachers. While some programs in teacher prepara- 
tion for open education are currently offered, the relative values of such pro- 
grams will apparently depend upon their success in rendering explicit the goals 
and objectives of open education and they complement and differ from those of 
traditional education. Most models of open education seem to place more emphasis 
upon methodology than upon objectives, whereas it would seem more appropriate for 
methodology to follow objectives. Even the few available instruments to assess 
educational Openness seem more materials- and method-oriented than goal-oriented. 
Much In need is a clearer statement of the behavioral objectives of open educa- 
tion. 

2, The transition from a traditional program to a more open program 
may be difficult for many students, requiring new levels of self-discipline and 
perhaps of self-confidence as well. New habits of study and of pacing will be 
needed. In return for his greater freedom, the student will have to learn to 
work efficiently in a sometimes chaotic environment, and he will have to learn 
to depend more upon himself and less upon his teachers for control and guidance. 
These skills and habits are not emphasized in traditional education; hence the 
open classroom teacher well may have to teach things largely unrelated to the 
academic curriculum. And it well may be that development of the skills and 
habits necessary to survival and continued academic progress are the distinctive 
characteristics and most important products of open education. Further, it seems 
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likely that the more prolonged the student's experience fn traditional educa- 
tion, the morci Important and the more difficult the acquisition of these skills 
and habits wi 11 be. 

3. Little attention seems to have been paid to the potential influ- 
ences of school personnel not directly involved in open education projects. 
Probably the assistance, support, and encouragement of administrators and other 
school personne) are more critical to the effectiveness of an innovative pro- 
ject than is commonly realized. Educators, like other humans, are frequently re- 
luctant to accept innovation, part i cular ly i f they are not direct participants* 
Similarly, the Innovation participants can easily feel threatened by a possi- 
bility of failure or isolated from accepted practices, and in such circumstances 
they may regress to behaviors Inappropriate to the i.ew situation. Because of 
its self-proclaimed variance from tradi tional educat Ion, open education must re- 
cognize that teachers, too, require guidance and leadership. 

The open education program evaluated in this study will continue, and eval- 
uation of it will also continue. The research model incorporated into the de- 
sign of this study has proved its value, and as the concepts of open education 
are more closely realized in the program, this model will continue to be employed 
in evaluation of the merits of open education,^ 
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Table 2. 

Intercorrelatlons of OSCAR Teachers* 
Mean Ratings of Students 

Social 

Hath. Studies Science 

Language .k27 MO .395 

Mathematics A96 .86} 

Social Studies .516 
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